Toward a Formalization of QA Problem Classes

How tough is a given question-answering problem? Answers to this question differ greatly among different researchers and groups. To begin rectifying this, we start by giving a quick, simple, propaedeutic formalization of a question-answering problem class. This formalization is just a starting point and should let us answer, at least roughly, this question: What is the relative toughness of two unsolved QA problem classes? 1 Formalization of Question-Answering Problem Classes It is not an exaggeration to say that the AI community had a watershed moment when IBM’s Watson beat Jeopardy! champion Ken Jennings in a nail-biting match in 2011. QA is important to AGI: Levesque et al. [7] give an argument in defense of QA being a test for AGI/AI. Despite the importance and quite impressive real-world success of QA research, there is very sparse formalization on what makes a QA problem difficult. (See [4] for a formalization of a test for AI which has a QA format.) Concisely, our position is that: 1) QA is crucial for AGI; 2) a rigorous formalization of QA is important to understand the relative toughness of unsolved problems in QA; and finally 3) a formal understanding of QA is important for AGI. Toward this end, we start with a simple formalization that could point the way. We now present a simple formalization of a QA problem class. A QA problem class consists of a set of questions, a set of answers, a corpus, and some other computational artifacts. The formalization lets us judge, at least coarsely, whether one QA problem (e.g. Jeopardy! ) is tougher than another (e.g. answering queries about financial data).